Optical modulation formats supporting the need for high spectral-efficiency optical communications with high receiver sensitivity are attractive to future free-space and terrestrial optical communications. Optical differential multi-level phase-shift keying (ODMPSK) is a modulation format type that has attained high spectral efficiency. An optical differential 8-level phase-shift keying (OD8PSK) format with 3 bits/symbol has been proposed. (See, e.g., C. Kim et al., “Direct-detection optical differential 8-level phase-shift keying (OD8PSK) for spectrally efficient transmission,” Optics Express vol. 12, pp. 3415-3421, 2004). The combined use of ODMPSK and pulse amplitude modulation (PAM) has also been proposed to obtain high spectral efficiency. A 4-bit per symbol (or 16-state) modulation format based on differential quadrature-phase-shift keying (DQPSK) and a 4-level pulse amplitude modulation format (PAM) have been demonstrated. (See, e.g., K. Sekine et al., “Proposal And Demonstration Of 10-G symbol/sec, 16-ary (40 Gb/s) Optical Modulation/Demodulation Scheme,” Proceedings of European Conference on Optical Communication (ECOC'04), paper We3.4.5, 2004). Unfortunately, theses types of modulation formats suffer from a number of drawbacks.
More specifically, a major drawback of these types of modulation formats is poor receiver sensitivity due to their inefficient symbol constellations. Generally speaking, the minimum distance among the symbols in a constellation diagram of a modulation format, for a given average signal power, is desired to be maximized in order for the signal to have high a immunity to noise. In addition, detection of ODMPSK and ODMPSK/PAM is very complex. For ODMPSK, highly stabilized optical demodulators are needed. Each demodulator is also required to be able to compensate for the change in relative phase between its interfering arms resulting from ambient temperature change or signal carrier frequency drift.